Field of the Invention
The present invention relates to an image processor, image processing method and program, and an imaging system for connecting input images formed by a lens system.
Description of the Related Art
There is a known omnidirectional imaging system which includes multiple wide-angle lenses such as fisheye lens or super wide-angle lens to capture an image in omnidirections at once. It is configured to project images from the lenses onto a sensor surface and combine the images through image processing to thereby generate an omnidirectional image. For example, by use of two wide-angle lenses with angle of view of over 180 degrees, omnidirectional images can be generated.
In the image processing a partial image captured by each lens system is subjected to distortion correction and projection conversion on the basis of a certain projection model with a distortion from an ideal model taken into account. Then, the partial images are connected on the basis of an overlapping portion of the partial images to form a single omnidirectional image. The positions at which subject images overlap in the overlapping portion are detected by pattern matching or the like.
However, partial images with a large amount of distortion, for example, ones captured with the fisheye lens, contain connecting areas having different kinds or amounts of distortion even when the same subject is captured. Accordingly, it is very difficult to accurately detect the overlapping positions of the images by pattern matching. Thus, partial images cannot be connected properly and a high-quality omnidirectional image cannot be generated accordingly.
There are various known techniques to combine partial images captured with multiple cameras. For example, Japanese Patent Application Publication No. 2010-130628 (Reference 1) discloses an imaging device comprising partial cameras having an overlapping photographic area and capturing a partial area of a photographic area of a subject and a reference camera having a photographic area including a part of an image captured by each partial camera. It corrects a distortion of a captured image of each camera using a camera parameter, detects an image area in which the corrected images of the partial cameras and the corrected image of the reference camera coincide with each other, calculates a relative position, and connects the images.
Further, Japanese Patent Application Publication No. 2009-104323 (Reference 2) discloses a camera system which uses multiple cameras arranged to have overlapping photographic areas and generates a high-precision mapping table not to cause a displacement in connecting points according to an actually captured image without the estimation of set positions of the cameras. Japanese Patent Application No. 2010-81479 (Reference 3) discloses an image processor which converts only X-coordinates of a fisheye image of a vehicle on the road captured with an in-vehicle camera, to generate a virtual view image with no vanishing points. It intends to convert fisheye images into images such that parking lot lines on a road surface appear to be approximately linear and parallel.
However, Reference 1 relates to connecting images represented in plane coordinates and cannot detect connecting positions accurately when applied to an imaging device using a lens with a large distortion such as a fisheye lens. Also, Reference 2 teaches the generation of mapping tables by use of an existing target board but cannot align the positions of images accurately. Reference 3 teaches the correction of fisheye images but does not concern connecting a plurality of images.